Spontaneous scale-free structure of spike flow graphs in recurrent neural networks

نویسندگان

  • Filip Piekniewski
  • Tomasz Schreiber
چکیده

In this paper we introduce a simple and mathematically tractable model of an asynchronous spiking neural network which to some extent generalizes the concept of a Boltzmann machine. In our model we let the units contain a certain (possibly unbounded) charge, which can be exchanged with other neurons under stochastic dynamics. The model admits a natural energy functional determined by weights assigned to neuronal connections such that positive weights between two units favor agreement of their states whereas negative weights favor disagreement. We analyze energy minima (ground states) of the presented model and the graph of charge transfers between the units in the course of the dynamics where each edge is labeled with the count of unit charges (spikes) it transmitted. We argue that for independent Gaussian weights in low enough temperature the large-scale behavior of the system admits an accurate description in terms of a winner-take-all type dynamics which can be used for showing that the resulting graph of charge transfers, referred to as the spike flow graph in the sequel, has scale-free properties with power law exponent gamma=2. Whereas the considered neural network model may be perceived to some extent simplistic, its asymptotic description in terms of a winner-take-all type dynamics and hence also the scale-free nature of the spike flow graph seem to be rather universal as suggested both by a theoretical argument and by numerical evidence for various neuronal models. As establishing the presence of scale-free self-organization for neural models, our results can also be regarded as one more justification for considering neural networks based on scale-free graph architectures.

برای دانلود رایگان متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Spontaneous Scale-free Structures in Spike Flow Graphs for Recurrent Neural Networks

In this thesis we present both theoretical and numerical results, revealing self-organization in graphs of functional connections for a number of recurrent neural network models. The consequence of studied self-organization is the spontaneous emergence of scale-free connectivity. The concept of the scale-free network (whose vertex degree distribution obeys a power law) discussed in detail in th...

متن کامل

Spectra of winner-take-all stochastic neural networks

In Piekniewski & Schreiber (2008) we have developed a simple mathematical model for information flow structure in a class of recurrent neural networks and shown that its asymptotic behaviour is scale-free and admits a description in terms of the so-called winner-take-all dynamics. In the present paper we establish a limit theorem for spectra of the spike-flow graphs induced by the winner-take-a...

متن کامل

Spectra of the Spike Flow Graphs of Recurrent Neural Networks

Abstract. Recently the notion of power law networks in the context of neural networks has gathered considerable attention. Some empirical results show that functional correlation networks in human subjects solving certain tasks form power law graphs with exponent approaching ≈ 2. The mechanisms leading to such a connectivity are still obscure, nevertheless there are sizable efforts to provide t...

متن کامل

Spectra of the Spike-Flow Graphs in Geometrically Embedded Neural Networks

In this work we study a simplified model of a neural activity flow in networks, whose connectivity is based on geometrical embedding, rather than being lattices or fully connected graphs. We present numerical results showing that as the spectrum (set of eigenvalues of adjacency matrix) of the resulting activity-based network develops a scale-free dependency. Moreover it strengthens and becomes ...

متن کامل

A biologically plausible learning rule for the Infomax on recurrent neural networks

A fundamental issue in neuroscience is to understand how neuronal circuits in the cerebral cortex play their functional roles through their characteristic firing activity. Several characteristics of spontaneous and sensory-evoked cortical activity have been reproduced by Infomax learning of neural networks in computational studies. There are, however, still few models of the underlying learning...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

عنوان ژورنال:
  • Neural networks : the official journal of the International Neural Network Society

دوره 21 10  شماره 

صفحات  -

تاریخ انتشار 2008